[Firebase AI] Add workaround for invalid SafetyRatings in response#14817
[Firebase AI] Add workaround for invalid SafetyRatings in response#14817andrewheard merged 5 commits intomainfrom
Conversation
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses a bug in the backend that causes invalid SafetyRating values to be returned. The changes include adding an unspecified case to the HarmCategory, HarmProbability, and HarmSeverity enums, updating the SafetyRating initializer to handle missing category and probability values, and filtering out invalid SafetyRating values in the Candidate struct. Additionally, integration tests have been re-enabled for Vertex AI configurations, and new unit tests have been added to verify that invalid safety ratings are ignored. Overall, this is a well-structured solution to the problem.
Summary of Findings
- Handling of unspecified values: The introduction of
unspecifiedcases forHarmCategory,HarmProbability, andHarmSeverityand the subsequent handling in theSafetyRatinginitializer is a good approach to gracefully handle potentially missing or invalid data from the backend. - Filtering of safety ratings: The filtering of safety ratings within the
Candidatestruct ensures that invalid ratings are not propagated, preventing potential issues in downstream processing. - Test coverage: The re-enabled integration tests and the new unit tests provide good coverage for the changes, ensuring that the workaround functions as expected and does not introduce regressions.
- Documentation: The comments added to explain the workaround and the purpose of the
unspecifiedcases enhance the code's readability and maintainability.
Merge Readiness
The pull request appears to be in good shape for merging. The changes address the identified bug effectively, include adequate testing, and are well-documented. I am unable to approve the pull request, and recommend that others review and approve this code before merging.
2 participants
Added handling for empty
SafetyRatingvalues in responses. This may occur during image generation usinggemini-2.0-flash-expwhere the backend (Vertex AI only) returns additional emptySafetyRatingobjects in thesafetyRatingsarray (i.e.,{}, {}, {}, {}). See FirebaseExtended/vertexai-sdk-test-data#37 for more details.